Members: Mengning Mao(5683130), Yiming Wang, Yao Lu(5665043), Yuzhou Lu(5855241), Yi Yang(5717981)
Date: 08-11-2022
How has COVID-19 impacted transportation in the United States?
Author 1: Mengning Mao - Background research, all analysis of Q5, Q7
Author 2: Yiming Wang - Background research, all analysis of Q3
Author 3: Yao Lu - Background research, all analysis of Q1, Q8
Author 4: Yuzhou Lu - Background research, all analysis of Q4, Q6
Author 5: Yi Yang - Background research, all analysis of Q2
import pandas as pd
import math
import scipy
import numpy as np
import datetime
import geopandas as gpd
import json
import itertools
from urllib.request import urlopen
%matplotlib inline
import matplotlib.pyplot as plt
import seaborn as sns
import plotly.express as px
import plotly.graph_objects as go
import plotly.io as pio
from plotly.offline import init_notebook_mode
import matplotlib.animation as animation
init_notebook_mode(connected=True)
pio.renderers.default = "plotly_mimetype+notebook"
with urlopen('https://raw.githubusercontent.com/plotly/datasets/master/geojson-counties-fips.json') as response:
counties = json.load(response)
# Read worldwide COVID data
# https://ourworldindata.org/coronavirus/country/netherlands
covid_world = pd.read_csv('https://raw.githubusercontent.com/yangyi1141/TIL-LabAssignments-Group16/9bd1ff225fc7d0f9993e3c6481a75c34711641f1/owid-covid-data.csv')
# Daily Average Number of People Staying at Home
df = pd.read_csv('Trips_by_Distance.csv')
# Read data of passenger vehicles to US from Canada
# (https://www.bts.gov/covid-19/week-in-transportation)
df_ca = pd.read_csv('https://raw.githubusercontent.com/yangyi1141/TIL-LabAssignments-Group16/main/Passenger_Vehicles_to_US_from_Canada_data.csv')
# Read data of passenger vehicles to US from Mexico
# (https://www.bts.gov/covid-19/week-in-transportation)
df_me = pd.read_csv('https://raw.githubusercontent.com/yangyi1141/TIL-LabAssignments-Group16/main/Passenger_Vehicles_to_US_from_Mexico_data.csv')
# Get Percent Change in Passenger Travel from 2019 by Mode
df_passenger = pd.read_csv("https://raw.githubusercontent.com/yangyi1141/TIL-LabAssignments-Group16/main/Passenger_data.csv")
# Read data of the number of confirmed cases of COVID-19 in the USA
# (https://ourworldindata.org/explorers/coronavirus-data-explorer)
df_covid = pd.read_csv('https://raw.githubusercontent.com/yangyi1141/TIL-LabAssignments-Group16/main/owid-covid-data-2.csv')
# Read data of transit ridership of different transit modes
# (https://data.bts.gov/Research-and-Statistics/Monthly-Transportation-Statistics/crem-w557/data)
df_pt = pd.read_csv('https://raw.githubusercontent.com/yangyi1141/TIL-LabAssignments-Group16/main/Monthly_Transportation_Statistics.csv')
# Get Transportation Services Index and Seasonally-Adjusted Transportation Data
# (https://data.bts.gov/Research-and-Statistics/Transportation-Services-Index-and-Seasonally-Adjus/bw6n-ddqk/data)
df_service = pd.read_csv("https://raw.githubusercontent.com/yangyi1141/TIL-LabAssignments-Group16/main/Transportation_Services_Index_and_Seasonally-Adjusted_Transportation_Data(2).csv")
# Get Total flight depatures by week:2019,2020,2021,2022
df_depart = pd.read_csv("https://raw.githubusercontent.com/yangyi1141/TIL-LabAssignments-Group16/main/Total_Flight_Departures_by_Week_Total_Flights_data.csv")
Firstly, we want to look at the situation of the COVID-19 pandamic in the United States from the start to most recent dates.
Within a complete dataset of different factors of the COVID-19 situation, the possible data that we can extract from it could be the nunber of new cases, new deaths etc, which can be related to and match with the transportation data (that we will produce later) based on common dates.
# Extract the United States' COVID data
covid_usa = covid_world[covid_world.location == 'United States']
# Reset index and drop unnecessary columns
covid_usa = covid_usa.reset_index()
covid_usa = covid_usa.drop(columns = ['index', 'iso_code','continent'])
# Convert the dates to datetime format
covid_usa_date = pd.to_datetime(covid_usa['date'], format='%Y-%m-%d')
covid_usa['date'] = covid_usa_date
# Fill NaN values with zero
covid_usa = covid_usa.fillna(0)
# covid_usa.head()
# Process the data so that the dates match with the dates in Part 4: Freight Transportation data
# Find the monthly mean covid cases
covid_usa['month'] = covid_usa['date'].dt.month
covid_usa['year'] = covid_usa['date'].dt.year
covid_usa['day'] = 1
covid_usa['monthYear'] = pd.to_datetime(covid_usa[['month', 'year', 'day']])
mean_cases = covid_usa.groupby(['monthYear'])['new_cases_smoothed'].mean().to_frame(name = 'Cases').reset_index()
mean_cases.columns = ['DATE', 'Monthly mean new cases']
# Find the monthly mean covid deaths
mean_deaths = covid_usa.groupby(['monthYear'])['new_deaths_smoothed'].mean().to_frame(name = 'Deaths').reset_index()
mean_deaths.columns = ['DATE', 'Monthly mean new deaths']
# Merge the monthly mean cases and deaths
cases_deaths = pd.merge(mean_cases, mean_deaths, on="DATE")
The outbreak of COVID-19 pandemic started in January 2020, which was roughly two years and half to our latest data time (July 2022). The number of new cases and deaths both have fluctuated significantly during this time.
The overall trend of people diagnosed with the coronavirus during 2021 is an almost constant increase, with a few ups and downs in between. The highest number of new cases in 2021 was around 200,000 persons per day. Although the daily number of new cases does not fluctuate too much and seems to be decreasing recently (July 2022), the sudden increase in January 2022 was rather drastic, equalling almost 3 to 4 times of the relative peak values in other months.
On the other hand, the number of new deaths kept altering throughout the whole observation period. Important coincidences with the case observation to notice are that similar to new cases, the peak and valley data points of deaths occurred in January 2022 and July 2021 as well. Note that the scale of the two diagrams vary. While the number of new cases a day can be up to almost 800,000, the maximum daily deaths is less than 3,500 persons.
fig = px.line(covid_usa, x='date', y='new_cases_smoothed')
# Label the diagram
fig.update_xaxes(title_text='Date', showgrid = False)
fig.update_yaxes(title_text='Cases (persons)')
fig.update_layout(title_text = 'Daily New COVID-19 Cases in the United States (2020-2022)')
fig.update_traces(line_color='blue', line_width=3)
# Store as a json file
fig.write_json('q1_1.json')
# Show figure
fig.show()
fig = px.line(covid_usa,x='date', y='new_deaths_smoothed')
# Label the diagram
fig.update_xaxes(title_text = 'Date', showgrid=False)
fig.update_yaxes(title_text = 'Deaths (persons)')
fig.update_layout(title_text = 'Daily New COVID-19 Deaths in the United States (2020-2022)')
fig.update_traces(line_color='red', line_width=3)
# Store as a json file
fig.write_json('q1_2.json')
# Show figure
fig.show()
# Data processing
# Transfer to pandas time
df.loc[:, 'Date'] = pd.to_datetime(df.loc[:, 'Date'])
df['Year'] = df['Date'].dt.year
# Calculate percentage of people staying at home
df['Percentage Staying'] = 100 * df['Population Staying at Home'] / (df['Population Staying at Home'] + df['Population Not Staying at Home'])
# Get national data in 2019, 2020, 2021
df_n = df.groupby('Level').get_group('National')
n19 = df_n.groupby('Year').get_group(2019)
n20 = df_n.groupby('Year').get_group(2020)
n21 = df_n.groupby('Year').get_group(2021)
# Get weekly mean value
perc19 = n19.groupby('Week')['Percentage Staying'].mean()
perc20 = n20.groupby('Week')['Percentage Staying'].mean()
perc21 = n21.groupby('Week')['Percentage Staying'].mean()
# Create a new dataframe for weekly mean value
perc = pd.DataFrame(data={
'2019':perc19,
'2020':perc20,
'2021':perc21}
)
# Get county data
df_c = df.groupby('Level').get_group('County')
# Select the data of April 2020
c20 = df_c.groupby('Year').get_group(2020)
c21 = df_c.groupby('Year').get_group(2021)
apr20 = c20.groupby('Month').get_group(4)
nov21 = c21.groupby('Month').get_group(11)
mean1 = apr20.groupby('County FIPS')['Percentage Staying'].mean()
dict1 = {'County FIPS': mean1.index,'Percentage Staying': mean1.values}
map1 = pd.DataFrame(dict1)
# Select the data of November 2021
mean2 = nov21.groupby('County FIPS')['Percentage Staying'].mean()
dict2 = {'County FIPS':mean2.index,'Percentage Staying':mean2.values}
map2 = pd.DataFrame(dict2)
# Line chart
fig = px.line(perc,markers = True)
fig.update_xaxes(title_text="Week during a year")
fig.update_yaxes(title_text="Percentage of People Staying at Home")
fig.update_layout(title_text="Weekly Average Pencentage of People Staying at Home: 2019–2021")
# Store as a json file
fig.write_json('q2_1.json')
fig.show()
# Heat map for Percentage Staying at Home April 2020
fig = go.Figure(data=go.Choropleth(
locations = map1['County FIPS'],
geojson=counties,
z = map1['Percentage Staying'],
zmin=0,
zmax=80,
colorscale = 'Viridis',
autocolorscale=False,
marker_line_color='darkgray',
marker_line_width=0.5,
colorbar_title = 'Percentage'
))
fig.update_layout(
title_text='Percentage Staying at Home April 2020',
geo_scope='usa', # limite map scope to USA
geo=dict(
showframe=False,
showcoastlines=False
)
)
# Store as a json file
fig.write_json('q2_2.json')
fig.show()
# Heat map for Percentage Staying at Home November 2021
fig = go.Figure(data=go.Choropleth(
locations = map2['County FIPS'],
geojson=counties,
z = map2['Percentage Staying'],
zmin=0,
zmax=80,
colorscale = 'Viridis',
autocolorscale=False,
marker_line_color='darkgray',
marker_line_width=0.5,
colorbar_title = 'Percentage'
))
fig.update_layout(
title_text='Percentage Staying at Home November 2021',
geo_scope='usa', # limite map scope to USA
geo=dict(
showframe=False,
showcoastlines=False
)
)
# Store as a json file
fig.write_json('q2_3.json')
fig.show()
The world of passenger travel changed dramatically with the arrival of COVID-19 in March 2020. The daily average number of Americans staying at home, normally between 58 and 68 million in 2019, jumped to 94.5 million in March 2020 and has remained high above the 2019 levels. It peaked at over 100 million during the 2020 holiday season.
The difference in the percentages of people staying at home varies greatly from county to county. The analysis of the line chart above shows that the number of people at home peaked in April 2019. In the second half of 2020, the number of people at home decreases significantly and stabilizes. Comparing the percentage of people at home in April 2020 with the percentage in November 2021 shows the change in numbers between counties in the US. The maps show that although there is variation between counties, the number of people at home in April 2020 is greater overall than the number of people at home in November 2021, echoing the line graph. This in turn suggests that the impact of the epidemic on people's travel starts around April 2020.
# Find the number of trips on the 1st of the month
df1 = df.iloc[[365,396,425,456,486,517,547,578,609,639,670,700]]
# The figure of number of trips in 2020
fig = go.Figure()
fig.add_trace(go.Scatter(x=df1["Date"], y=df1["Number of Trips <1"],mode='lines+markers',name='Number of Trips <1'))
fig.add_trace(go.Scatter(x=df1["Date"], y=df1["Number of Trips 1-3"],mode='lines+markers',name='Number of Trips 1-3'))
fig.add_trace(go.Scatter(x=df1["Date"], y=df1["Number of Trips 3-5"],mode='lines+markers',name='Number of Trips 3-5'))
fig.add_trace(go.Scatter(x=df1["Date"], y=df1["Number of Trips 5-10"],mode='lines+markers',name='Number of Trips 5-10'))
fig.add_trace(go.Scatter(x=df1["Date"], y=df1["Number of Trips 10-25"],mode='lines+markers',name='Number of Trips 10-25'))
fig.add_trace(go.Scatter(x=df1["Date"], y=df1["Number of Trips 25-50"],mode='lines+markers',name='Number of Trips 25-50'))
fig.add_trace(go.Scatter(x=df1["Date"], y=df1["Number of Trips 50-100"],mode='lines+markers',name='Number of Trips 50-100'))
fig.add_trace(go.Scatter(x=df1["Date"], y=df1["Number of Trips 100-250"],mode='lines+markers',name='Number of Trips 100-250'))
fig.add_trace(go.Scatter(x=df1["Date"], y=df1["Number of Trips 250-500"],mode='lines+markers',name='Number of Trips 250-500'))
fig.add_trace(go.Scatter(x=df1["Date"], y=df1["Number of Trips >=500"],mode='lines+markers',name='Number of Trips >=500'))
fig.update_xaxes(title_text="date")
fig.update_yaxes(title_text="number of trips")
fig.update_layout(
title="The change in number of trips in 2020",
autosize=False,
width=1000,
height=600,
)
# Store as a json file
fig.write_json('q3_1.json')
fig.show()
# Find the number of trips on the 1st of the month
df2 = df.iloc[[0,31,59,90,120,151,181,212,243,273,303,334]]
# The figure of number of trips in 2019
fig = go.Figure()
fig.add_trace(go.Scatter(x=df2["Date"], y=df2["Number of Trips <1"],mode='lines+markers',name='Number of Trips <1'))
fig.add_trace(go.Scatter(x=df2["Date"], y=df2["Number of Trips 1-3"],mode='lines+markers',name='Number of Trips 1-3'))
fig.add_trace(go.Scatter(x=df2["Date"], y=df2["Number of Trips 3-5"],mode='lines+markers',name='Number of Trips 3-5'))
fig.add_trace(go.Scatter(x=df2["Date"], y=df2["Number of Trips 5-10"],mode='lines+markers',name='Number of Trips 5-10'))
fig.add_trace(go.Scatter(x=df2["Date"], y=df2["Number of Trips 10-25"],mode='lines+markers',name='Number of Trips 10-25'))
fig.add_trace(go.Scatter(x=df2["Date"], y=df2["Number of Trips 25-50"],mode='lines+markers',name='Number of Trips 25-50'))
fig.add_trace(go.Scatter(x=df2["Date"], y=df2["Number of Trips 50-100"],mode='lines+markers',name='Number of Trips 50-100'))
fig.add_trace(go.Scatter(x=df2["Date"], y=df2["Number of Trips 100-250"],mode='lines+markers',name='Number of Trips 100-250'))
fig.add_trace(go.Scatter(x=df2["Date"], y=df2["Number of Trips 250-500"],mode='lines+markers',name='Number of Trips 250-500'))
fig.add_trace(go.Scatter(x=df2["Date"], y=df2["Number of Trips >=500"],mode='lines+markers',name='Number of Trips >=500'))
fig.update_xaxes(title_text="date")
fig.update_yaxes(title_text="number of trips")
fig.update_layout(
title="The change in number of trips in 2019",
autosize=False,
width=1000,
height=600,
)
# Store as a json file
fig.write_json('q3_2.json')
fig.show()
# Find the number of trips on the 1st of the month
df3 = df.iloc[[731,762,790,821,851,882]]
# The figure of number of trips in 2021
fig = go.Figure()
fig.add_trace(go.Scatter(x=df3["Date"], y=df3["Number of Trips <1"],mode='lines+markers',name='Number of Trips <1'))
fig.add_trace(go.Scatter(x=df3["Date"], y=df3["Number of Trips 1-3"],mode='lines+markers',name='Number of Trips 1-3'))
fig.add_trace(go.Scatter(x=df3["Date"], y=df3["Number of Trips 3-5"],mode='lines+markers',name='Number of Trips 3-5'))
fig.add_trace(go.Scatter(x=df3["Date"], y=df3["Number of Trips 5-10"],mode='lines+markers',name='Number of Trips 5-10'))
fig.add_trace(go.Scatter(x=df3["Date"], y=df3["Number of Trips 10-25"],mode='lines+markers',name='Number of Trips 10-25'))
fig.add_trace(go.Scatter(x=df3["Date"], y=df3["Number of Trips 25-50"],mode='lines+markers',name='Number of Trips 25-50'))
fig.add_trace(go.Scatter(x=df3["Date"], y=df3["Number of Trips 50-100"],mode='lines+markers',name='Number of Trips 50-100'))
fig.add_trace(go.Scatter(x=df3["Date"], y=df3["Number of Trips 100-250"],mode='lines+markers',name='Number of Trips 100-250'))
fig.add_trace(go.Scatter(x=df3["Date"], y=df3["Number of Trips 250-500"],mode='lines+markers',name='Number of Trips 250-500'))
fig.add_trace(go.Scatter(x=df3["Date"], y=df3["Number of Trips >=500"],mode='lines+markers',name='Number of Trips >=500'))
fig.update_xaxes(title_text="date")
fig.update_yaxes(title_text="number of trips")
fig.update_layout(
title="The change in number of trips in 2021",
autosize=False,
width=1000,
height=600,
)
# Store as a json file
fig.write_json('q3_3.json')
fig.show()
By comparing the data on the number of people's trips between 2019, 2020 and 2021, we can see the impact of COVID-19 on people's travel. First, analyze the distribution of the number of trips. More than half of people have 0 to 3 daily trips. The number of trips from 3 to 5, 5 to 10 and 10 to 25 is basically the same, and their combined share is more than 40%, and only a few people make more than 25 trips per day. In 2019, the number of people's trips remained in a relatively stable range. The overall trend is that the number of trips in the middle of the year is slightly higher than that at the beginning and end of the year. When the outbreak of COVID-19 hit in March 2020, the number of people's travel dropped significantly and has remained at a relatively low level compared to 2019. In 2021, the number of people's trips will gradually pick up.
# Change column type to pandas date time
df_ca.loc[:, 'Date'] = pd.to_datetime(df_ca.loc[:,'Date'])
df_ca = df_ca.set_index('Date')
# Define a figure
fig = px.line(
df_ca,
x=df_ca.index,
y=df_ca.columns
)
# Labeling the axes of the diagram
fig.update_xaxes(title_text='Time')
fig.update_yaxes(title_text='Passenger Vehicles')
# Adding title to the diagram
fig.update_layout(
title_text='Comparison of Passenger Vehicles to US from Canada before and during COVID-19'
)
# Store as a json file
fig.write_json('q4_1.json')
# Show the figure
fig.show()
# Change column type to pandas date time
df_me.loc[:, 'Date'] = pd.to_datetime(df_me.loc[:,'Date'])
df_me = df_me.set_index('Date')
# Define a figure
fig = px.line(
df_me,
x=df_me.index,
y=df_me.columns
)
# Labeling the axes of the diagram
fig.update_xaxes(title_text='Time')
fig.update_yaxes(title_text='Passenger Vehicles')
# Adding title to the diagram
fig.update_layout(
title_text='Comparison of Passenger Vehicles to US from Mexico before and during COVID-19'
)
# Store as a json file
fig.write_json('q4_2.json')
# Show the figure
fig.show()
Canada and Mexico border the northern and southern parts of the U.S., respectively, so observing the changes in the number of their cross-border vehicles can deduce the impact of COVID-19 on the travel of passenger vehicles in the northern and southern parts of the U.S.
It can be seen that before COVID-19, which is the baseline in the figure, the number of passenger vehicles transiting between Canada and the U.S. fluctuated between 400k and 700k, and when COVID-19 emerged, there was a significant drop in the number of those, reaching a minimum value of even less than 50k. And then, until August 2021, the number of transit vehicles had not risen significantly and remained at a low level. However, after that, the number of transit vehicles began to increase and recover towards the pre-epidemic situation, although it still did not end up where it was before.
The number of transit vehicles between Mexico and the U.S. has been showing an overall upward trend after the same significant drop as in the north at the beginning of COVID-19. Around April 2022, the number of passenger vehicles crossing in the south has reached its previous value, and even slightly more than before, which differs dramatically from the recovery of the northern border crossing. Overall, the similarity between the changes in transit vehicles on the northern and southern borders is that initially the number of transit vehicles on both borders dropped significantly due to the travel restrictions imposed on people entering both borders during the highly contagious phase of COVID-19; for example, travel for tourism, shopping, and routine family visits were not allowed. The difference between them is that traffic on the southern border has declined significantly less than that on the southern border, and has recovered faster and better, which is due to more cross-border employment near the southern border.
# Change column type to pandas date time of Percent Change in Passenger Travel from 2019 by Mode
df_passenger.loc[:, "Day(Week Ending)"] = pd.to_datetime(df_passenger.loc[:, "Day(Week Ending)"])
df_passenger
fig = px.line(
df_passenger,
x=df_passenger.loc[:, "Day(Week Ending)"],
y=df_passenger.loc[:, "Week Change"],
color="Mode"
)
# Labeling the axes of a diagram
fig.update_xaxes(
title_text="Time",
showgrid=False # No grid lines are drawn for the x-axis
)
fig.update_yaxes(title_text="Change from baseline")
fig.update_layout(
title_text="Percent Change in Passenger Travel from 2019 by Mode" #Adding title to the chart
)
# Store as a json file
fig.write_json('q5.json')
# Show figure
fig.show()
COVID-19 has led to a sharp decline in direct travel for both long-distance and local passenger traffic. Two of the long-distance trips, air and rail transport, were hit the hardest but recovered at a high rate. With the exception of personal vehicle travel, which has largely recovered, recovery is lagging, especially for transit and intercity buses, which have only recovered by about half.
# Process ata of confirmed cases in the USA
# Extract valid data of confirmed cases in the USA for the first day of each month
df_covid = df_covid[df_covid['location']=='United States']
df_covid = df_covid.loc[217666:218547,['date','new_cases']]
df_covid.index = range(len(df_covid))
df_covid = df_covid.iloc[[0,29,60,90,121,151,182,213,243,274,304,335,366,394,425,455,
486,516,547,578,608,639,669,700,731,759,790,820,851,881],:]
# Redefine the order of index
df_covid.index = range(len(df_covid))
# Expand the value of the number of confirmed cases to a value range similar to that of riders
df_covid['new_cases'] = df_covid['new_cases'] * 1000
# Process data of riders for different public transits in the USA
df_pt.loc[:, 'Date'] = pd.to_datetime(df_pt.loc[:,'Date'])
df_pt = df_pt.loc[877:906,['Date','Transit Ridership - Other Transit Modes - Adjusted',
'Transit Ridership - Fixed Route Bus - Adjusted','Transit Ridership - Urban Rail - Adjusted']]
df_pt.index = range(len(df_pt))
# Combine the dataframes
df_com = pd.concat([df_covid,df_pt], axis=1)
df_com = df_com.drop('Date', axis=1)
df_com = df_com.set_index('date')
# Define a figure
fig = px.line(
df_com,
x=df_com.index,
y=df_com.columns
)
# Labeling the axes of the diagram
fig.update_xaxes(title_text='Time')
fig.update_yaxes(title_text='Number of people')
# Adding title to the diagram
fig.update_layout(
title_text='Relationship Between Confirmed Cases and Different Transit Modes'
)
# Store as a json file
fig.write_json('q6.json')
# Show the figure
fig.show()
It can be seen that the public transport modes that are mainly affected by the number of confirmed cases, i.e. the severity of COVID-19, are fixed bus and urban rail, while the other transit modes are quite less used and therefore less affected. For both bus and rail, the overall trend is almost the same, with a significant drop in ridership at the beginning of COVID-19 and a slow increase thereafter. It can be found that the troughs in ridership are around the time when the number of confirmed cases peaks, so it can be said that the number of passengers on public transportation is a good indicator of the severity of COVID-19. The difference between riderships of the two modes is that at the beginning of COVID-19, the number of rail transit users dropped more sharply, indicating that rail transit is more sensitive to the epidemic.
# Rename columns
df_service = df_service.rename({'OBS_DATE': 'DATE'}, axis=1)
# Change column type to pandas date time of Percent Change in Passenger Travel from 2019 by Mode
df_service.loc[:, "DATE"] = pd.to_datetime(df_service.loc[:, "DATE"])
# Remove the Weekly Total Flights item that duplicates every year
df_depart_new = df_depart.drop_duplicates(subset=["date (International) Week","date (International) Year"])
fig = go.Figure(data=[
# Unadjusted passenger transportation data series
go.Scatter(name="Unadjuested", x=df_service.loc[:, "DATE"], y=df_service.loc[:, "RPM"]),
# Seasonally-adjusted passenger transportation data series
go.Scatter(name="Seasonally-adjusted", x=df_service.loc[:, "DATE"], y=df_service.loc[:, "RPM_D11"])
])
# Labeling the axes of a diagram
fig.update_xaxes(title_text="Year")
fig.update_yaxes(title_text="Air Revenue Passenger Miles(RPMs)")
# Show vertical and horizontal line (crosshair) as hoverinfo
fig.update_xaxes(spikemode="across")
fig.update_yaxes(spikemode="across")
fig.update_layout(
title="Monthly Changes in Air Travel for Equivalent Periods", #Adding title to the chart
hovermode="x unified" # Make hover messages display two data points
)
# Store as a json file
fig.write_json('q7_1.json')
# Show figure
fig.show()
fig = px.line(
df_depart_new,
x=df_depart_new.loc[:, "date (International) Week"],
y=df_depart_new.loc[:, "Weekly Total Flights"],
color="date (International) Year"
)
# Labeling the axes of a diagram
fig.update_xaxes(
title_text="Month",
showgrid=False # No grid lines are drawn for the x-axis
)
fig.update_yaxes(title_text="Total Flights Departures by Week: 2019,2020,2021,2022")
fig.update_layout(
title_text="Airline Travel as Measured by Airport Screenings as a Proxy: 2019–2022", #Adding title to the chart
hovermode = "x" , # Make hover messages display 4 years of data at the same time
xaxis = dict(tickmode='array',
tickvals=df_depart_new.loc[:, "date (International) Week"], # Original value
ticktext=['Jan','','','','Feb','','','','Mar','','','',
'Apr','','','','May','','','','','Jun','','','',
'Jul','','','','','Aug','','','','Sep','','','',
'Oct','','','','','Nov','','','','Dec','','',''], # Final displayed value (replaced value)
tickangle=0 # Make ticks horizontal for easy reading
)
)
# Store as a json file
fig.write_json('q7_2.json')
# Show figure
fig.show()
In December 2005, the Bureau of Transportation Statistics published a report on “Airline Travel Since 9/11.” It came four years after the devastating attack caused a tragic loss of life and a significant contraction in U.S. air travel. The 20-year anniversary of 9/11 coincides with another devastating event, the COVID-19 pandemic, that is also affecting air travel.
As the charts above show, the immediate impact of both events was a severe drop in the number of passengers traveling by air with one key difference. For the same number of months before and after the onset of the disruption, the drop in monthly passengers in September 2001 from the previous month is significantly less steep (33%) than the drop in 2020 (96%).
All air service in the United States was suspended on 9/11, but the aviation system was restored within days. Passenger travel by commercial airlines did not recover until March 2004 when the number of passengers enplaned returned to the August 2001 level.
In contrast, the COVID-19 pandemic disrupted aviation service over many months through a patchwork of travel restrictions among states and foreign destinations. By June 2021 (15 months after the initial lock-downs in March 2020), air travel had recovered to the point that 82% of the passengers (61.6M) carried in June 2019 (87.3M).
As the chart above shows, the number of departing flights each week since May 2020 (2020 in orange, 2021 in green, and 2022 in purple) has been inching toward the number of flights in the corresponding week in 2019 (blue).
Freight transportation remains an important sector of the transportation industry. This part of analysis aims to process the data on freight transportation in the U.S. to reflect changes in the demand for goods and services before and after the outbreak of the COVID-19 pandemic.
# Read total transportation data of the United States
# https://data.bts.gov/Research-and-Statistics/Transportation-Services-Index-and-Seasonally-Adjus/bw6n-ddqk
transport_usa = df_service
# Filter for days that match the COVID data of the United States
transport_usa = transport_usa[(transport_usa['DATE'] > '2019-12-01') & (transport_usa['DATE'] < '2022-11-01')]
# Extract freight data by dropping irrelevant columns
freight_usa = transport_usa.drop(columns = [
'ID', 'ASM', 'ASM_D11', 'ASM_D', 'ASM_D_D11', 'ASM_I', 'ASM_I_D11', 'RPM', 'RPM_D11', 'RPM_D', 'RPM_D_D11',
'RPM_I','RPM_I_D11', 'LOAD_FACTOR', 'LOAD_FACTOR_D11', 'LOAD_FACTOR_D','LOAD_FACTOR_D_D11',
'LOAD_FACTOR_I', 'LOAD_FACTOR_I_D11','ENPLANE_I', 'ENPLANE_I_D11', 'ENPLANE_D', 'ENPLANE_D_D11',
'ENPLANE', 'ENPLANE_D11','AIR_RPM_TSI', 'AIR_RPM_TSI_D11', 'RAIL_PM', 'RAIL_PM_D11', 'TRANSIT',
'TRANSIT_D11', 'PETROLEUM', 'PETROLEUM_D11', 'NATURAL_GAS', 'NATURAL_GAS_D11', 'VMT','VMT_D11',
'TSI_Passenger', 'TSI_Passenger_C', 'TSI_Freight', 'TSI_Freight_C', 'TSI_Total', 'TSI_Total_C',
'IND_PRO', 'MANUF','IDX_AIR_RPM', 'IDX_AIR_RTMFM', 'IDX_RAIL_FRT_CARLOADS','IDX_RAIL_FRT_INTERMODAL',
'IDX_RAIL_PM', 'IDX_TRANSIT_D11', 'IDX_PETROLEUM_D11', 'IDX_NATURAL_GAS_D11', 'IDX_WATERBORNE_D11',
'IDX_TRUCK_D11', 'AIR_RTMFM', 'RAIL_FRT_CARLOADS', 'RAIL_FRT_INTERMODAL', 'WATERBORNE', 'INV_TO_SALES'
])
# Rename column names
freight_usa.columns = ['DATE', 'Air Freight (Ton Miles)', 'Rail Freight', 'Rail Intermodal', 'Waterborne (Million Tons)', 'Truck Tonnage Index']
# Fill NaN values with zeros
#freight_usa = freight_usa.fillna(0)
# Rename columns
freight_usa = freight_usa.rename({'OBS_DATE': 'DATE'}, axis=1)
# Convert data type to float
freight_usa = freight_usa.replace(',','', regex=True)
freight_usa.iloc[:, 1:7] = freight_usa.iloc[:, 1:7].astype(float)
# freight_usa.head()
# Process the freight data for ease of visualisation later
freight_usaNew = freight_usa.iloc[:, 0:6].melt(id_vars=["DATE"],
var_name="Type",
value_name="Number")
# Merge the processed covid data and the freight data
covid_freight = pd.merge(cases_deaths, freight_usa, on="DATE")
# Find correlation matrix within the merged data
matrix = covid_freight.corr(
method = 'pearson', # The method of correlation
min_periods = 1 # Min number of observations required
)
# Store the merged data
covid_freight.to_csv('covid_freight.csv', encoding='utf-8', index=False)
Freight transportation remains an important sector of the transportation industry. This part of analysis aims to process the data on freight transportation in the U.S. to reflect changes in the demand for goods and services before and after the outbreak of the COVID-19 pandemic.
From the diagram, it can clearly be seen that unlike passengers, the quantity of goods being transported in all types did not experience a sharp decrease during the COVID-19 pandemic. Rather, most of the freight transportation modes increased during this time. For instance, the air freight increased significantly from mid-2020 to the start of 2022, though it kept decreasing slightly until July 2022.
For the two modes of rail freight, the changes are not that apparent, however they both have increased as the pandemic lasted, after they initially decreased in the first few months of the pandemic. The tonnage of truck freight seems to be rather unchanged throughout 2020 to 2022, as the numbers only fluctuated very slightly.
# Plot the different freight transport data based on dates in a line plot
# Differentiate the lines by the freight type
fig = px.line(
freight_usaNew, x="DATE", y="Number", color="Type",
hover_data=['Number']
)
# Labeling the axes of a diagram
fig.update_xaxes(title_text='Date',showgrid=False)
fig.update_yaxes(title_text='Load')
fig.update_layout(title_text='Monthly Freight Transportation in the United States (2020-2022)')
fig.update_traces(line_width=3)
# Store as a json file
fig.write_json('q8.json')
fig.show()
Correlation coefficients indicate the correlation between different variables, whose magnitude:
From the correlation coefficient matrix diagram, it can be seen that there are no very high or high correlations between two variables, other than the variable and itself. However, there can be seen some almost as high or moderate correlations, as well as low correlations between the freight transportation and the COVID-19 cases and deaths datasets.
For new COVID-19 cases, there is a moderate correlation between it and the new deaths (0.51). Regarding its correlations with freight, rail and truck freight only have a very low correlation with it (0.19 and 0.22, respectively). On the other hand, for new COVID-19 deaths, the waterborne and truck freight are slightly negatively correlated with it (-0.27 and -0.2, respectively). Interestingly, air freight has a low correlation with both new cases and deaths of COVID-19, which are 0.37 and 0.34, respectively.
The correlations between the different freight transportation are mostly moderate or low. Air freight is moderately correlated with intermodal rail freight (0.61) and somewhat correlated with normal rail freight (0.46). Rail freight is moderately correlated with both the intermodal rail (0.56) and truck tonnage, whilst having a weak correlation with waterborne freight. Besides those, truck and waterborne tonnage are also moderately correlated.
plt.figure(figsize = (16,5))
plt.title('Correlation for COVID-19 and Freight Transportation in the United States',fontsize = 20)
matrix_new = covid_freight.corr().round(2)
sns.heatmap(matrix, annot=True, vmax=1, vmin=-1, center=0, cmap='vlag')
plt.show()